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PREFACE 



William W . Turnbull 



Both professional end public attention is focust,d these 
days on the concept of educational accountability. The word 
"accountability" has as many meanings as people care to give it 
and is often used in connection with such activities as assess- 
ment, evaluation, auditing, and performance contracting. 

So great is the interest — and so meager the clarifica- 
tion of the many issues and problems involved — that the need for 
a comprehensive look at the concept prompted Educational Testing 
Service to sponsor a series of conferences on accountability. 

We are indeed fortunate that some of the most knowl- 
edgeable and thoughtful people concerned with the philosophy, 
strategies, and pit f 11s of accountability in education accepted 
our invitation to participate in this conference, Each of them 
lias provided a substantive and challenging contribution to better 
understanding of what is involved in developing and implementing 
accountability programs of integrity and merit. 

Because of the urgent need for dissemination of informa- 
tion about accountability the speakers' papers, in their pre- 
conference foim, have been assembled in this booklet for immediate 
distribution , 
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T H E M E A 11 S A N D ENDS OF 
A C C 0 V N 7 A B I L I T Y 

E r i ck L . Lin dman 



Benefits and Costs of Education 

A few years ago a hardy band cf program budgeteers gathered 
together the elements of their planning, prograrjHng, and budgeting 
sysiem, packed them fh a waterproof box, and departed from the 
Pentagon where they had spent many fruitful years. Their departure 
was without certmoiy. There seemed to be neither sadness nor Joy 
at their leaving; bl.t evidence of relief was clearly visible beneath 
the surface, 

Ihe sturdy band set sail for education land where they 
hoped to spread the gospel of PPBS. When they reached education 
land they were admitted without delay and were warmly welcomed, for 
they promised something for everyone. For school boards they 
promised more education for the school dollar. For administrators 
they promised a management information system with modern built-in 
feedback controls; and for teachers they promised that the budget 
vould no longer be the mysterious obstacle to their plans, but in- 
stead would become a helpful genie. 

Although these promises brought a warm welcome, the pro- 
gram budgeteers soon encountered difficulties. In order to carry 
out their highly advertised benefit-cost analyses, it was necessary 
to assess the benefits of various educational programs in terms of 
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dollars so that these benefits could be compared with costs incurred 
in achieving then. But no one in education land was willing ai.d 
able to place dollar values upon the benefits to individuals and to 
society of various educational accomplishments. 

It was during this period of frantic search for a way to 
assess the benefits of education that our brave budgeteers met 
another crusading group , known as the behavioral objectivists. Un- 
like the program budgeteers who were mostly economists, the behavioral 
objectivists were experts in educational measurement. They believed 
that if a child learns, his behavior will change and it is possible 
to measure the amount of the change by suitable tests- From this 
simple proposition they concluded that educational goals and objec- 
tives should be stated as expected changes in student behavior and 
that teaching effectiveness should be measured by changes in student 
behavior . 

Although the program budgeteers and behavioral objec- 
tivists started their reform movements separately, it was inevit- 
able that they should discover each other and find that they had 
much in common. The program budgeteers thought they had at last 
discovered someone who could evaluate the educational product and 
put a dollar value upon it so they could make their benefit-cost 
analyses . The behavioral objectivists, on the other hand, were not 
especially interested in the mysteries of the budgetary process, but 
they liked the idea of getting a little bit closer to the source of 
money. Besides, by forming a union, both crusadirg groups were 
better able to protect themselves from the hostile humanists whose 
attacks were gi*ovi ng in number and intensity every day. So the two 
refoxm groups worked together, sharing the same platform and seeking 
converts t? their cor men cause. 

After a short period of cooperation, disagreements began 
to emerge. The program budgeteers seemed obsessed with costs and 
the amount of resources needed for various programs and goels. 'ihey 
were interested in the budgetary process culminating in the allocation 
of resources to various educational programs. The behavioral objec- 
tivists, however, were interested in instructional planning and 
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evaluation, and the!]' activities cutninated in the selection of a 
teaching procedure. 

Both the program budgeteers and behavioral objecM v is: .1 
found it impossible to place dollar values upon the beha/Lirnl 
objectives which had been formulated, For example, they wr* Ui able 
zo determine the value to society of teaching all children Iogod 
aspects of Korn on history. Moreover, they were unable to del' r: line 
the cost of achieving various behavioral objectives beca n:-e th- 
cost depended upon the methods used arid the ease with viieh ti.** 
students learn. 

It became increasingly clear that there are inner' '-v 
distinctions as well as interrelationships between ( 1 ) a .■"nvj.vnt 
system designed to a llocate resources among various cducati :.'il 
programs arid {^) an instructional planning and evaluation system 
designed to util i ze educational resources most effectively . 

Moreover, instructional planning and evaluation are part 
of the ongoing teaching process and therefore should continue to 
function after the budgetary decisions have been made for a fiscal 
period. Clearly , the two systems function on different tir.c 
sehedules. The budgetary process is necessarily geared to the 
fiscal year while the instructional planning pnd evaluation prec-nss 
is, or should be, continuous* 

Anothei distinction stems from the divergent concerns of 
the two systems. The budgetary process is primarily con erned with 
derisions that affect costs such as class size, salary schedules, 
the maintenance of summer schools, and so fort! , whereas the* in: - mo- 
tional planning and evaluation process is concerned primarily will, 
finding more effective teaching ’-rocedurus and techniques which, oft a 
have no effort upon annual costs. 

These distinctions y.nke it i ncreasi ngly clour that thi j 2 
tv.'in budgctcei o and * he Tchaviora] object] v i rts snoul d ernera'", Ini 
they should remain s jffiri* ntly independent ro that eu?h can maim 1 \ic 
unique contribution to education. 



T h e Kac ten -y A n a 1 c gy 

One of the inherent v aknc sses of * he art ieuini r 




Mr tendency to \ ror \ : c more than he can dcizv-r. 
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years ago the authors of the seven cardinal principles 
accepted as a school's responsibilities such goals as: 
wort hv use of leisure time, ethical character, worthy home member- 
ship, and civic responsibility es well as command of the fundamental 
processes. In response to these glowing promises school attendance 
and school budgets increased substantially, yet ve seem uc be no 
close, to these goals than ve were in 1918. We promised more than 
we were able tc deliver and since the public expected more, its 
disappointment has been great. 



pointing out that :i f we had used behavioral objee 4 ives in 1918 when 
ve wrote the seven cardinal objectives ve wouldn't be in this 
trouble today. Th:is is perhaps true, but there is also a danger 
of promising more Ilian can be delivered in student performance. 

Unen students are regarded as "products” of the school, it is 
implied tnat the school is a factory and should be fully account- 
able for the behavior of its products. Yet, without the power to 
select its raw material and reject defective products, the school 
cannot guarantee its product . 



as a product of his high school implies more responsibility than 
the school car accep>t. His behavior is influenced by many factors 
beyond lie control of his school . The time-honored allies of the 
home and school ;n the chi ld-rearing process -- the church, the 
Y.'CA , the Boy Scouts, the Camp Fire Girls -- are still with us, 
tut they are losing their c zYcet 1 ver>ess . So is Horatio Algt 's 
message to youth that hard work and dedication bring success. 

Along with the deter iorat icn of these faroilar gui deposts 
is the triumphal entry of television. The new Pied Piper is pre- 
sorting mo v e and ir.-re of the waking hours of children and youth, 
providing a powerful distraction from the school’s three R's, which 
have lev h'ieisen balings. Unfortunately , thi° twentieth century 
i 1 i e 1 piper spends a d i s prop ort i on at * amount of t ir.e selling soap 
r-:,d cereals. 

With all of these factors contributing either positively 
or negatively to the education e 4 ' children arid youth, to call ^ ^on 



The behavioral objectiv: sts concur with this diagnosis, 



To regard an l3-year old youth in the graduating class 
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the products of the school is to promise more than the school is 
able to deliver. The factory analogy leads inevitably to a demand 
for a quality control system in which all enrollecs of the school 
are expected to perform- at a preestablished level of skill as 
measured by appropriate tests. The great diversity of human talent 
and varying contributions of the home and peer groups makes this 
degree of uniformity impossible. 

If the factory analogy and the guaranteed product are 
rejected, for what should che schools then be held accountable? 

Trie answer, it seems to me, is quite clear. The school should be 
held accountable for the scope and quality of the educational 
services it renders to ius students. For example, a school may 
veil guarantee the following educational services for its students: 
(l) No first-grade pupil shall be in a class with mo than 25 
pupils, (2) All pupils whose reading score? are more than one year 
below their grade norm snail receive 15 minutes of individual 
tutoring each day, ( 3 ) Sumner classes shall be available for any 
student who failed a high school course required for graduation. 
Other statements describing the scope and quality of educational 
services could be added. 

These statements emphasize the educational services to 
be available to the student. For these services the school can 
and should be held accountable. Moreover, the effectiveness of 
these services in contributing to student test performance and to 
subsequent successful living should be under constant review, tut 
even with such checks upon the* school's ef fecti veness , it is im- 
possible to guarantee the test score of an individual student in 
the same way that a manufacturer guarantees hi; product. For these 
reasons, then, the schocl must be accountable for the scope and 
quality of educational services it renders to students even though 
it cannot guarantee an individual student's test performance as 
implied by the factory c-ialogy. 

It will be noted that although the scope and quality of 
educational sej vices rendered to students is an output of the school 
system, not an input, it does provide essential information eded 
for estimating program costs, lor this reason, the i dent i f i cat i n 
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of the scope and quality of services to be rendered is especially 
relevant to the budgetary process. By way of contrast, the be- 
havioral objectives lack cost relevance. It is not possible to 
determine the coot of teaching a child the multiplication tables. 
The cost is a function of the method used to achieve the behavioral 
objective, not of the objective itself. Thus, while the evalua- 
tion process appropriately emphasizes behavioral objectives , the 
budgetary process must be concerned with the scope and quality of 
the educational services to be rendered to students. 



Benefit-Cost Relationships 

The inherent difficulties encountered in placing a dollar 
value upon behavioral objectives make benefit-cost analysis in the 
true sense ol the word difficult, if not impossible, in the field 
of education. However, significant cost comparisons can be made if 
either the goals or the amount of resources used are held constant. 

For example, two different ways to achieve the same 
educational objective might be tried. If the same goal is achieved 
by both methods, a surong argument can be made for accepting the less 
costly of these methods. In this analysis alternative ways to achieve 
established educational goals are compared, h^cice that a dollar 
value to society of the goals has not been established. 

Another kind of analysis is possible in which the amount 
of resources e /.pended is held constant but these resources are used 
in different ways to achieve unequal educational outcomes. In this 
analysis, the educational procedure which achieved the greatest educa- 
tional output would, of course, be preferable. Such rest effectiveness 
studies can be made because they do net call for placing a dollar 
value upon the educational benefit itself. 

In a real sens? these two analyses indicate a funda- 
mental ch«' is occurring in American education at this time. 

When 1 fir. t started teaching, back in the early thirties, the 
superintendent of schools made speeches to the chamber cf commerce 
about getting the most for the school dollar. He didn't talk about 
} I }v> or about benefit-cost relationships, bet he did talk about 
getting the most for the school dollar. Hie approach reflected the 
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prevailing notion that tie needs of education are unlimited but 
the number of dollars which society can allocate to education are 
limited. He wanted to get as much money as possible for the schools 
and then to spend it wisely y maximizing the amount of educational 
service rendered. 

The basic assumption under lying his approach to budgeting 
-- that the needs of education are unlimited -- is being questioned 
today. Under the concepts of both the program budgeteers and the 
behavioral object ivists , limited educational goals are established 
first, then the least costly way to achieve them is sought. In 
the past, it was assumed that . V .oals of education were indeed 
unlimited ana the problem was ir get the most education for the 
available school dollars. We are now being asked to specify 
rather precisely limited objectives for education and then search 
for the most economical way to achieve them. In one sense loth 
procedures are the same, since t.ucy both seek the most cost-effoctiw 
use of school dollars. Their differences are more subtle. 

Under the unlimited educational goal concept, boards of 
education and professional educators determine educational prv.rilios 
after the legislature and the taxpayers have determined the amount of 
funds to be available. Under the limited educational goal concept, 
the legislature and the taxpayeis leviev and approve the educational 
goals and then provide sufficient funds to attain them. 

Along with these changes in the locus of the goal-setting 
function, hard-pressed taxpayers are challenging the unlimited goal 
doctrine. Specifically, they are asking: "Hnv can ve tell when 

too much is being s . nt on public education?" 

Ihis is a question that educators usually have avoided. 

V r nen an answer was given, it often emphasized unmet, educational needs. 
Such mi answer is no longer acceptable since there are unmet need 5 
in every phase of American life. To answer the question, the inhe- 
rits of another million dollars for public schools mm t be compared 
with the benefits of an additional million dollars for welfare, for 
defense, for ] dilution control, and so on. 

ii is l^nds inevitably to value judgments which are always 
involved when important educational issue* are resolve :i . At id this is 

/$ 
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vhere the humanists are likely to ambush the program budgeteers and 
ebavioral objeetivists . The ensuing conflict will pit the superior 
statistical skills of the program budgeteers and behavioral objee- 
tivists against superior rhetorical skills of the humanists. The 
outcome is in doubt. 

Regret ably, this encounter will be based upon the assump- 
tion that PPBS and behavioral objectives are, per se, good or bad 
for education. The truth is, they can be either. The right kind 
of each in the right quantities could be just the tonic our ailing 
school system needs; an overdose of either could be fatal. 
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ISSUES IN IMPLEMENTATION 

I 



Mark R. Shedd 



I would like to begin by stating the obvious. Account- 
ability in education, parti :ularly in public education, is long 
overdue: No longer can anyone involved in the education of our 

young people, whether it be the superintendent of schools, the 
district superintendent, the principal, the clessroa.i teacher or 
the teacher’s aide, be classified as a sacred cow — above reproach 
and immune from having to prove to the public that he is doing his 
job an d doing it veil. 

Accountability may be a tough word for many educators to 
swallow, but I'm convinced that the actual digestion of the 
accountability process in education will be much less painful than 
the first bitter taste. Of course, there is still a question or 
two t) be answered; such as, what i_s_ "the accountability process?" 

To answer uhat question, I'd like to take a brief look 
backward at the educational scene since the passage of the 
Elementary and Secondary Education Act of 19^5. 

Since then, ve have a: s nation poured billions of dol- 
lars into c oxz e n s at cry e d u cat I*. t h rough out t h e c ou n t ry . We h a v e 
looked to the iv/ults of those efforts and v;e have found letter 
schools, more libraries, more paraprofessional personnel , higher 
paid teachers and administrators, and higher tax burdens. 
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Yet, in the face of all of these improvements, there is 
no major study that shows at this time any significant improvement 
in pupil learning, reduced dropout rates or better teaching. Un- 
like professionals in medicine we are not able to say that our 
clients measurably are better for the effort. 

One of the few things we can prove conclusively is that 
education can consume large sums of money with few measurable 
achievements to show for it. 

This record, particularly as it is seen through the eyes 
of those outside education, raises some pretty basic questions of 
accountability. For instance, Governor Russell Peterson, of 
Delaware, speaking before the Education Commission of the States 
last July in Denver, put it very bluntly when he said: 

"Educators traditionally think in terms of inputs--nev 
programs, more dollars for materials, higher teacher salaries and 
the like. We have files, and wastebaskets, full of statistics 
about education — how many schools, how many teachers, how many 
strikes and campus rebellions. 

"In principle, the American educational commitment has 
been that every child should have an adequate education, but when 
a child fails to learn, school personnel have all too often 
labeled him 'slow, 1 'unmotivated' or 'retarded.' Our schools 
must assume the commitment that every child shall learn...." 

Nolan Estes, the superintendent in Dallas, speaking 
before one of Educational Testing Service's previous accounta- 
bility conferences, described his own situation by saying: 

"Our target had been those schools in which students 
were averaging only a half-year's achievement gain for every 
full scholastic year. By the time we finished, we had not man- 
aged to improve on this sad record; in fact, some of our Title I 
schools were worse off in l ( /fO than they had been in 19^5 • 

"Five years and five billion dollars after Title I was 
passed," he added, "We still have not learned how to break the 
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cycle of underachievement that sees children from poor homes do 
poorly in school; find poor jobs or none; marry — and then send 
their ovn poor children to school." 

The stark fact remains that, after years of experi- 
mentation, after spending millions upon millions of dollars, and 
after applying some of the most sophisticated technological and 
human skill in the world to the problem, ve continue to grind out 
thousands of functional illiterates from the educational mill 
each year. 

It is certainly no wonder that the nublic does net 
believe it is getting its money's worth from public education and 
has begun to beat the drums for accountability. 

Yet, there is no magic "accountability" button that one 
can push , nor is there any one segment of the public educational 
structure that can bear tne brunt of accountability. 

Traditionally, as Fred Hechinger, education editor of 
the New York Times points out, it has been the student who has 
been held accountable for his perfoimance in school. "The class- 
room," Hechinger contends, "has been the teacher's castle, and 
schools have largely escaped che burden of responsibility for any 
lack of student achievement." 

Then, with the advent of massive federal remedial educa- 
tion programs in the mid- ' 60 s, the ringer began to point at the 
home as not holding up its share of educational c ccountability , 

Now, as we embark on the decade of the '70s, it has be- 
come the "in" thing to do to make the teacher the whipping boy for 
the breakdown in the educational process. The public, angered by 
escalating educational costs, primarily due to lai'ge increases in 
pay eacn year for teachers, has grabbed at the concept of account- 
ability and is attempting to use it as a whip against teachers. 

This I object to very strongly. Ac eoun lability should be 
a process by which we can show ourselves how to ueach better and 
h (V to turn out a better educated child, not as? a weapon to beat a 
dead horse. 



Account ability will get nowhere, as it most certainly 
should not, if it is cast in the punitive sense of saying uo a 
teacher: M Here, you SOB, raise all these kids' achievement levels 

a certain rate or you'll get fired. 1 ' 

Accountabilj ty must involve everyone from the super- 
intendent of schools to the classroom aide. It must also involve 
the parent and business and industry as veil. And it must involve 
processes as well as people. 

Terrel Bell, USOE deputy commissioner for school systems, 
put the whole matter of accountability in proper perspective when 
he told \ recent ETS conference: 

1T Ac countability looks at school resource deployment , 
materials selection, time allocations, and a host of other school 
management practices. Needless to say, accountability has many 
facets, forms, and faces. It reaches far beyond the simplistic 
assertion that it is concerned with teachers and teaching. When 
the students fail to learn, the entire system must be introspective. 1 ’ 

With + his in mind, a look at the experiment in New York 
City involving ETS and the Board of Education is most heartening. 
Briefly, the New York Board has contracted with ETS to come up with 
an accountability design to define performance objectives for both 
students arid staff, with the ultimate goal of recommending an 
administrative structure for an accountability system. 

What is heartening is the assurance of everyone concerned 
that the system will be a positive rather than negative one, aimed 
not at threatening job security, but at making it possible for 
ever v one to do a better Job. 

With that assurance, the New York teachers union, the 
United Pederetion of Teachers, has backed the plan. Union President 
Albert Shanker joined in the announcement with the Board, proclaiming 
that the accountability system would ’’provide the greatest protection 
teachers have e\oi % known.” 

He explained this by saying that successful teachers would 
be protected against unfair criticism, by providing proof of th^ir 
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effectiveness, while the system would indicate the additional train- 
ing needed by ineffective teachei s . 

I would certainly consider this a workable agreemer* , one 
which would help dispel the fear among professionals of the word, 
"accountability , " 

Yet, it ironic that militant teacher organizations vho 
had spoken so strongly against accountability formulas until the 
New York experiment , nay themselves have been one of the reasons 
for the accountability clamor. 

Traditionally the public has viewed teachers and school 
adminstrators as practicing professionals, and has placed a high 
degree of trust in the professional judgment of school personnel* 
That, however, was before collective bargaining and teacher strikes. 
It was not long before the public, conditioned to the 
fact that teachers had chosen to deal with their employers in the 
same way employees negotiated with large industrial concerns, 
began to ask: "What do we get in return?" — in much the same way 

that corporate management asks how many more or better cars or 
television sets will be produced as a result of the contract 
settlement , 

Paradoxically, school personnel are insisting that the 
public continue to trust them as professionals in the same way 
that physicians and attorneys are trusted* Just as an example, 
few patients would think of trying to negotiate a guaranteed cure 
with a doctcr* Rather, ve expect the doctor to exercise skill and 
professional judgment in treating our ills. 

Educators have, however, negotiated a different managerial 
ethic than that enjoyed by other professionals. It may be a big 
step in the right direction, simply because the outcome of the 
negotiations has been accountability* 

Eut I think I've talked enough about the reasons for 
accountability, I'd like now to get, briefly, to three of the 
accountability situations that have developed in J'hiladelphi a , 

Two in/olve guaranteed performance contracts, alout which I won't 




say very much because I'm sure other speakers at this cor. ference 
will delve deeply into this aspect of accouritabi lity . The third 
involves our own home-grown brand of accountability applied to a 
district-wide reading program. 

For the record, we do have the nation's largest guaran- 
teed performance contract under which the Behavioral Research 
Laboratories, in return for $600*000, has guaranteed to raise the 
reading achievement levels of 15 >000 pupils by one grade level in 
one year's time on a money back basis for every pupil who doesn't 
make it. The tests have already been gwen, and we'll know’ in a 
few short weeks Low ve have done. 

The other performance contract is with the Westinghouse 
Learning Corporation, through the nationwide, 20-school district 
UJoK project, involving 600 students in reading and math. Here, 
too, we’ll know shortly where we stand. 

but what I'd really like to talk with you about is our 
eight-district reading program, involving all 285*000 students in 
grades K-12 throughout the city. In assessing our difficulties 
with teaching Johnny to read, it became painfully evident that we 
v'ere sadly lacking in an adequate delivery system--a system that 
would assure a plan's success by following a tough, predetermined 
set of goals and objectives based on a process of accountability. 

We began to devise this system, in conjunction with th'‘ 
personnel of our eight school districts, by planning eight dif- 
ferent attacks on the reading problem, each one geared to meet the 
specific needs of the children in that district. It did not come 
cosily , 

First, ve had to gear up our research capabilities to 
measure quality by output, rather than input. Educational research 
going back to the '20s always measured such things as expenditure 
i or pupil, the number cf library books, dollars spent on teachers 1 
salaries, and numbers and statistics almost down to the number of 
was h bowls per student. 
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Relatively few, if any, educational dollars were spent on 
charting the output, the actual achievement of the pupils. This we 
did through the application of standardized tests, and the results 
in Philadelphia, as in all the other major cities across the nation 
were devastating. 

To put it bluntly, ^0 percent of our students were 
functionally illiterate. 

Our first function was to develop and have approved by 
the Board of Education specific, no-nonsense, system-wide reading 
goals that would make us accountable to the pupils, their parents, 
and the public for reaching certain new plateaus of reading achieve 
ment in a given time period. 

This we did, and the goals are bold ones. I'd like to 
quote from the board resolution which set them up. 

"By the end of the school year 1975-76, each student 
receiving instruction in regular classes will acquire those read- 
ing skills necessary to adequately function in school and society; 
and students in Philadelphia will read as ve?.l or better ‘ban 
students elsewhere. 

"It is further resolved that the following indicat cr£ 
will be accepted as evidence that the goal has been accomplished: 

1) All students who leave the Philadelphia schools at 
age l6 or older shall read on the 8th grade level or above; and 

2) . The scores of all students r:i an appropriate test 
shall indicate that the percentage of students who read at graie 
level or above is greater than the national norm; and 

3) All children completing the sixth grade shall read 
at the 5*5 level or above." 

Admittedly, in light of past failures, these goals will 
be extremely difficult to meet , to say the least. But we honestly 
feel wo can do it. 

Cnee the goals were announced, we were faced irjr*e 15 at' ly 
with the task of cranking up a management system capable of attain- 
ing them. in effect, ve simply bu3 to build a better mousetrap. 
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We did, in fact . build a nev delivery system, and ve hope 
it's a better one. It consists of a technical task grcjp of reading 
experts whose job it vill be tc ride herd on the entire reading 
improvement program; a detailed statement of system-management needs 
to be followed; alternative implementation plans to throw into the 
breech until ve find one that works; and projections of tine, costs, 
and resource demands. 

But, of course, building this type of management delivery 
system left as wit}: two more problems: We had to develop new skills 

and management capabilities within existing schcol system personnel 
through staff and leadership development programs; and ve had to 
recruit top-level management persons from outside the school system, 
particularly in previously neglected areas such as data processing, 
planning, budgeting , finance and research. This, too, ve have 
accomplished . 

Finally, after undertaking all this extremely complex re- 
tooling, ve had to tie it all together if ve expected ic to work at 
al] . The thread ve used was accountability. 

V,e were firmly convinced that accountability in a large 
urban school system trying to teach children how to rea^ was no 
more difficult than accountability in a large industry trying to 
reach its own production goals. If business can hold its employees 
accountable for production, and attain .■ ccoss through that account- 
ability process, then a school system should be able to do the 
same thing. 

Cur ac couii t r.b i 1 i ty is a tvo-v ay s tree t . 

The Board and the Buy erinton dent not only are accountable 
to the public for reaching their predetermined reading g_.als, but 
they are also accc unt isle to their employees to deliver to them the 
toy plies, equipment and administrative support and knew- hov to 
ac c am p 1 i s / 1 t ) e t a sk . 

In turn, each district sup ri r i nter der.i is recount able for 
suiport to hie rcr.ding project manager, his principals, and his 
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teachers , as veil as being accountable to the Superintendent for 
producing results in his district. 

The principals are accountable to their teachers and to 
the district superintendent; and the teachers, of course, are 
accountable not only to the principals, but also to the youngsters 
they are teaching. 

It must be understood clearly that the teachers rely on 
district arid central office delivery of goods and services in order 
to carry out their tasks. Administrators in charge of delivering 
tner components are, therefore, accountable to the teachers. 

We hope, obviously, that with all this accountabi 1 ity 
bi,-' It into a nev, considerably more sophisticated management de- 
livery system, ve can, at long lasu , produce concrete results in 
the form of measurable gains in student achievement and attitudes. 

We have, in essence, ,c 5tablished a hard-to-reach goal, 
handed each individual district the ball, supplied them with the 
very best coaching possible, and told then to run like hell. 

Ar.d ran they have* Tne districts have elected to use 
more than ^0 different app?*oach s to reading instruction, consis- 
tent with the needs of their individual youngsters. 

Besides involving B1L in the guaranteed performance con- 
tract, the districts are working with McGraw-Hill, SRA, Reader's 
Digest, KDL, Macmillan, Lippi n oit, Audi son-Ves ley , Random House, 
arid many more, on various va> s to teach reading to children with 
various needs. 

Tne firms arc supplying materials and resources right 
from pr'sei ool through high, school to meet the reeding needs of 
the districts. Methods range from the traditional to the boldly 
experimental, front multi-lingual programs to computers, from pipe- 
lines into universities to sel f-cont ained units. 

In other words, we have tried to put it all together, 
tut cn a step by step, district by district, school by school, 
child by child basis. Ve arc uring project management techniques, 
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delivery systems, data processing, and just about everything ve 
could get our hands on— all tied in with accountability. 

Ve agree, as you do, that youngsters should, indeed, 
have the right to read. And ve agree further that, although 
proper teacher preparation for, and performance in, the inner 
city school vill continue to be critical measures in determining 
whether ve reach our goal, ve feel strongly that school manage- 
ment vill have to face up to its share of the burden, get up off 
its posterior, and crank up some programs that aren’t doomed by 
their very makeup before they even get to the classroom. 

For this, ve must be held accountable. 

Yet, through all the bi rth pains of accountability , and 
its attendant pressures, I hav<‘ a f*"ir, a strong fear, that marris 
vill take precedence o/er tin* kid.'. i.l.» ns* 1 1 vos . James Cass, 
education editor of Saturday v iy w , sum:: it up extremely veil 
when he writes : 

"As we focus increasingly tn pupil performance as a 
measure of teacher- effect! veness , it would be easy to forget the 
complexity of the learning process — that individual children are 
very diiferenc, that they learn different things at different 
rates, and that even the same child learns at a different rate 
at different times. 

"If, therefore, the laudable effort to improve class- 
room practice by assessing teacher and school effectiveness merely 
results in placing more intense and sophisticated pressure on the 
children to perform- the very principle vill be denied in practice, 
for if the concept r.ear.s anything* it is that the ultimate 
accountability must be to the children." 

To that 1 would add, "Amen. 11 And to you 1 would say, 
thank you. It’s been a pleasure to address you. 
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In speaking to this topic at other ETS conferences, the 
late Robert W. Locke of McGrav-Kill used as a subtitle, f, /.c count abi li t 
Yes, Performance Contracting, Maybe.” As his replacement on this 
occasion, 1 can both honor his memory and try to cariy out his vork 
by quoting from his remarks about performance contracting : 

Contrary to vhat you have read in the papers, I believe 
that rest education companies look upon performance 
contracting as an undesirable way of doing business. 

For companies with carefully researched programs and 
the competence to train teachers, it is not particularly 
risky because they knov vhat kinds of results they can 
achieve. Hovever, it tuts them in a straight Jacket that 
rakes r e r fcrr.anco cent rr- cling less desirable than the 
same vork done under a simpler contract. The reason vhy 
certain com?, ani t. s , sv;h as my cvn, have reseondcl to the 
recent surre of id* } ’s is simply that- they hn/e the pro- 
gram.? , they c an provide the services, and they are villing 
to take the risk ir nxer to pet the business. A customer 
if a customer. 

It is v or t Ik n ot i n p t i . at m a: <;.* 1 a r gc r e id veil- run c cm, ;c an 1 e s 




have not sou mint to vin i c r formaace contracts, either because 
they consider the risk too hi rh or si;. ply because they have 
reservations about their ability to x or form. the icquisite 
r e r v i c e j . 
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The companies that a_r_£ willing to make perfomance con- 
tracts — and perhaps all education companies -- would 
surely agree on these things: 

1. That results in education cannot be guaranteed. In 
the fall and winter in Texarkana there were some 
misguided claims about programs that could "produce 
grade- level independent readers and writers by the 
end of the first grade" or some such, but virtually 
all companies understand that intellectual processes 
cannot be guaranteed in the way that soupmakers 
guarantee cleanliness. 

2. They understand also the critical importance of 
having reliable data around which to construct 
contracts. More on this point later, but the 
lack of sufficient data 1 j probably the main 
reason why sorr^ responsible companies have been 
reluctant to make performance contracts. 

3. ■j’hey also recognize that performance contracts 
make more sense for innovative programs than for 
conventional ones. It is hardly worthwhile for 
either school districts or the companies to 
write such involved agree men to merely for t re- 
purchase of textbooks. It makes much move sense 
for the installation of complex new systems of 
instructicn for which the Icornii ; er v ire ament 
will i . uv e t o l e r o organize d •• • n d 1 1 . e t e a c h i n g a l a f f 
retiMinei. Inis may Ic tiic chief value of per- 
formance contract i ng , t ceruse innov'tivf’ instruc- 
tional programs are very difficult to install and 
yet. hold nuch promise for t } . e improvement of 

l ducat, i cti . 

I susi ?ct , in short, that the education companies have 
much the same general views of i - -r fee r.ance contracting ns 
t he s c hoo 1 r.y a t ms t hat v 1 sh t o i * i re t hem . 
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I believe that Bob Locke accurately reported on the con- 
cerns, the hopes, and the uncertainties of those considering 
performance contracting. Let me ask you to look at two issues 
that seem to me to lie behind our attitudes on this aspect af 
"accountability . " The first is related to public policy and the 
adoption of programs by public bodies. The second has to do with 
the state of the art of establishing educational objectives and 
measuring results. To what extent can public authorities and the 
private sector rely on our present knowledge and techniques? 

There are, of course, a good many other issues that 
coal d be discussed in this context. Consider, for example, the 
probable impact in the decade 1968-1976 of the fact that enroll- 
ment in the elementary schools will decrease by five percent at 
tna sane time that the Office of Education estimates a steady 
increase in costs. Educators and educational companies will find 
themselves in the same boat: legislatures and school boards will 

want some new explications for rising costs if the old arqmment 
of rising numbers ct; no longer be used. Tne best argument may 
be that rising costs bring better processes and better quality of 
learning in the schools. Tne mere fact that school authorities 
are trying out new ways of doing things may be a factor which may 
be helpful in persuading legislatures almost regardless of results 
rut it may also mean that performance contracts will be in trouble 
if the legislators and the lublic feel that they involve 1 higu" 
profits, oven though the profit might be quite commensurate with 
that accented for other businesses cf comtnralle risk. 

Cr consider another fact: that today the scncols dt m t e 

a very small j art of their annual ludgets to purchasing matt rials 
arid services directly relate: to instructional \ re-grams from out si 
the system, tne can argue about vhat should be included it. this 
category, but I doubt if the ; er cent age figure is likely to exceed 
f.h j credit ter annum, or something like } cr inpil jer year cn 
the nvor'ige , which is to be ccr.i afed to average j er-] ut : 1 ex; *;i. lit 
ft' r all i u r j os c- s in 1 9c 8 o f t g 1 . Inc ens i d e r i r . g i h e v i s d c m u f w i 
scale icrformancc contracting , therefore, responsible arlhcritie- 
edmatitn viil i -,v.* to take into account the cas ability cf the 
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so-called educational industry tc undertake, on any large scale, 
nationwide contracting which would involve a large increase in that 
percentage. For the truth is that the educational companies are 
not big business today. They do not have a large staff of experts 
in the management of educational programs, or of specialists in the 
various aspects of schooling. Few if any of them * r oul d be able, or 
perhaps willing, to mount large-scale, geographically dispersed, and 
high quality programs in a hurry, particularly in view of the pos- 
sibility of relatively low pi of it margins. This is especially true, 

I think, when one considers the two factor.: of public policy and 
the state o ? the art to which I now ash you to direct your attention. 

Questions of Public Policy a n d Legal Authority 

We have a deeply rooted tradition that the state has the 
obligation and the duty to provide an educational program at public 
expense. Though the state is the sovereign power in regard to 
schools, much policy-making power has been delegated to tne school 
boards elected locally. When pol i cy -maki ng functions are delegated 
to public officials or bodies, they may not be redelegated to a 
private entity. In all phases of performance contracting from tie 
contents of the Request for Proposals to the contract itself, the 
board must reserve the policy-making decision to itself and delegate 
only "ministerial" tunctions. The dividing line between these 
categories is obviously difficult to draw, and so far as 1 know 
there are few Judicial determinations to help us with this specific 
subject* There are many details to be considered, varying from 
state to state kqending upon the stale constitutions, statutes, 
rules, and regulations. «'.g long as tne contracts ar° experimental 
in nature or are to irccure consulting services for a district, 
there a; j enrs to be 3 io legal obstacle that cannot te overcome. 

} orform.anco contracts let to date — ranging from the 
Gary contract for taking over an entire school to contracts merely 
to improve the skills cf selected students in selected subjects -- 
have not , so far as 1 n/v, yet been challeng'd in the courts. If 
I er Torrance contracts t • come a standard tract ice, however, t ** ey 
surely will threaten the interests of some group who well right 
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resort to litigation. Your legal advisers and counsel for potential 
contractors can thread their ways through the legal problems and 
give appropriate advice so that performance contracts -- of one sort 
or another — can be legally entered J nto; but, vhethe ~ such contracts 
will meet the objectives of the districts entering in ,o then remains 
to be seen. 

One likely possibility is that the public contracting 
authority, in order to carry out its policy-making fur. ct ions, will 
have to specify in the contract the procedures, personnel standards, 
currieuluns , and relations for the social environment of the school 
so precisely that the performance contractor will in bis ’’ministerial 
functions have littie room to move around. State laws and Judicial, 
interpretations may limit the range within which innovative programs 
can be conducted. The irony of the situation is at once apparent: 

To rfonr.ar.ee contracts entered into with the intent of creating 
change and taking advantage of flexible operations nay end up in- 
flexible, This ray not be what the school system, had :n mind as an 
objective. And from the point of view of the performance contractor 
it obviously reduces the attractiveness of the arrangement from 
Loth the professional and the profitable point of view. Considered 
as a long-range business possibility, tightly controlled contracts 
(in the sense of limitation on personnel end permissible procedures), 
which may be subject to annual negotiation or cancellation, are not 
riocessari ly ajrealing, especially to a small-scme industry which 
v,ould have to spread itself across thousands of school districts in 
the Ld lit*.. d t* t a t c s . 



dues lions hetate d to_ tj .e ft a to of the Art 

H ijv veil ; er romance c*u'j actually l “ measured, is a technic a 
Issue, ;e.d a com; lex -u. d serious one at that. ft and arc i sod teste, 
like a ry ether measuring devices, are subject to error jf r.easur-. non ■ 
lni" f d, it is a far great* r relative orroi of measure mo .t * ' r,ji tnat 
usually found in the » nmi r.eer :nr field from which r erfoT. ar.ee con- 



tracting has teen b orr- v»-- i . '.no ccnseiuei.ee cf this large 
r.easvr'. ment is that individuals vno have l * r, chosen be 'ous 
scared lr.v on a ; r-.-tes t navv a very roc i chance of see n; tie 
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higher on a correlated posttest even if little real learning has 
taken place. In the ausence uf true experimental control groups or 
longitudinal measurement, regression toward the mean may work to the 
advantage of the performance contractor, if not the student. 

There is strong pressure currently to use criterion-referenced 
tests in place of standardized norm- referenced tests as measures of 
performance and achievement. These have the attractiveness of com- 
paring a student l s performance with a seemingly objective standard, 
instead of with the performance of other students. They also are 
likely to be more specifically tailored to the objectives and content 
of particular units of instruction than are most standardized tests. 

But there is no guarantee that a criterion-referenced test is any 
less subject to skill sampling errors than standardized tests, and 
eventually the standard of satisfactory performance on a criterion- 
referenced test depends on someone 1 s j udgme r.t of what is acceptable 
or desirable. Who shall make the judgment? Certainly not the 
performance contractor. 

I do not for a moment intend that this brief excursion 
into a few of the technical intricacies and uncertainties of testing 
obscure a very real need. Our educational system has been inequi- 
table in its distribution of resources to develop human potential. 

It has acted selectively on racial and economic bases. Hojefully 
one thrust of accountability is the intention to redress that 
select iveness . I would Loj e that in the process we do everything 
possible to assure that our objectives, procedures , and .est scores 
are what they purport to be: genuine credentials to educational, 

social, and economic rr.oMiitv, and not torpor ary passports to a 
1 a t e r , more- t rag i c 1 e t down . 

y t y conclusion on the technical issue at this point is 
that the standardized tests must be used at some point -- they’re 
the boot we have at the moment -- tut that the criteria of satis- 
factory { erforr.ane'* in a | erfcum.ai.ee contract need the most careful 
explication, and \ rot ally slrOd include folic v-up measure a beyond 
the life of a rroject cr n’T.nm. 

I no i i rue of t runs feral i 1 i ty gets to the heart of the 
functions of performance conit acti r.w . the turnkey f ur.ct icn :s 
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one that we hear much about, and the one that in many respects makes 
the most intuitive sense. It includes elements of installation, 
demonstration, and training. The contractor thus is apt to be (or 
should be) particularly interested in the contract planning process. 
Who is involved? How did they become involved? What is the long- 
range plan? What action is being token that gives some assurance 
that the school intends to commit itself to adopting the new program 
if it is successful? 

As far as I know, the turnkey function of performance 
contracting has yet to be shown to occur. And, in my opinion, if 
it does occur it will be to a large extent unpredictable, given 
the present state of cur knowledge and techniques . At the v e ry 
least it will be contingent upon success of the contract, and 
results of even the current wave are not in yet. 

1 am not trying to lead to the proposition that I see 
no prospects for transferability. The point I want to make is that 
it is extremely important for both tie school and the contractor 
jointly to think through the role pei formance contracts are really 
supposed to play in the plans and programs of the school, and to 
develop means of providing a continuity of good results. f ihc 
point ir.deed leads to another issue, that of feasibility. 

There are questions of feasibility from the viewpoint 
of the contractor, the school, and both together. 

Feasibility for the contractor is, I think, largely a 
matter of his personnel capabilities, and of how much 1 e knows 
about his ] reluct f books, machines, syllabi, techniques, and no enj 
-- what they can do, with whom, and under vhat conditions. And 
tills will of Course impend in large \ art on how veil they have t oen 
tried out and evaluated. Trite ns it sounds, if the contractor docs 
not have ext.esive aud reliable perfoir.ance data in his files, be* 
is courting nasty surt rises, both school and contractor need to 
review evidence of exi ected } . r romance carefully arid :n good fnitn 
I e fore ent o r i n g i nt o an up r«: er.ent . 

I think of feasibility from the point of view cf the school 
largely in future terms. hlvi<-usly tie' 1 school has little to lore 




